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ABSTRACT 


This study assesses two potential problems with respect to the reporting of Employment Insurance (EI) 
and Social Assistance (SA) benefits in the Survey of Labour and Income Dynamics (SLID): (a) under- 
reporting of the monthly number of beneficiaries; and (b) a tendency to incorrectly report receiving 
benefits throughout the year, while in fact benefits may have been received only in certain months, 
leading to artificial spikes in the January starts and December terminations of benefit spells (seam 
effect). The results of the analysis show the following: 


(1) 


(2) 


(3) 


(4) 


The rate of under-reporting of EI in SLID is about 15%. Although it varies by month (from 0% to 
30%), it is fairly stable from year to year. 


There are significant spikes in the number of January starts and December terminations of EI 
benefit spells. However, the spikes in January starts appear to represent a real phenomenon, 
rather than a seam problem. They mirror closely the pattern of establishment of new EI claims 
(the latter increase significantly in January as a result of the decline in employment following the 
Christmas peak demand). There are no corresponding statistics for EI claim terminations to 
assess the nature of December spikes. 


The rate of under-reporting of SA in SLID is about 50%, significantly greater than for El. The 
rate of under-reporting goes down to about 20% to 30%, if we assume that those who received 
SA, but did not report in which months they received benefits, received benefits throughout the 
year. 


There are large spikes in the number of January starts and December terminations. As in the case 
of EI, the SA could reflect a real phenomenon. After all, SA starts and terminations are affected 
by labour market conditions, in the same way EI starts and terminations are affected. However, 
the SA spikes are much larger than the EI spikes, which increases the probability that these, at 
least in part, are due to a seam effect. 


Keywords: SLID; under-reporting; seam effect; Employment Insurance; Social Assistance 
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1. Executive Summary 


The objectives of this study are to assess two potential problems with respect to the reporting of 

Employment Insurance (EI) and Social Assistance (SA) in the Survey of Labour and Income 

Dynamics (SLID), and propose ways of overcoming them: 

¢ under-reporting of the number of beneficiaries and amount of benefits received: 

¢ a ‘seam effect" which arises from a tendency to incorrectly report receiving benefits 
throughout the year, while in fact benefits may have been received only in certain months, 
leading to artificial spikes in the January starts and December terminations of benefit spells 


(seam effect); this problem distorts the length of benefit spells and the sequencing of different 
types of benefits. 


2. Findings 
a) Employment Insurance 


The results of the analysis show the following: 
EI under-reporting: 


e The rate of under-reporting of EI in SLID is about 15%. Although it varies by month (from 
0% to 30%), it is fairly stable from year to year. 

e The extent of under-reporting of EI benefits payments in SLID is about 10%, relative to the 
Human Resources Development Canada (HRDC) administrative data. 


EI seam effect: 


e There are significant spikes in the number of January starts and December terminations of EI 
benefit spells. 

e However, the spikes in January starts appear to represent a real phenomenon, rather than a 
seam problem; they mirror closely the pattern of establishment of new EI claims, which 
increase significantly in January as a result of the decline in employment following the 
Christmas peak demand. 

e The spikes in December terminations also most likely represent a real phenomenon. The 
spikes can be explained by the rise in seasonal jobs in December, resulting in EI beneficiaries 
finding employment and leaving EI. However, because we do not have administrative data to 
compare with, it is not clear to what extent the December spikes represent a real phenomenon 
and to what extent a seam problem. 
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b) Social Assistance 


SA under-reporting: 


@ The rate of under-reporting of SA in SLID is about 50%, significantly greater than for EJ. The 
rate improves to about 20% to 30% if we assume that, those who received SA, but did not 
report in which months they received benefits, received benefits throughout the year. 


e The amount of SA benefits reported in SLID is about two-thirds of the amount of provincial 
administrative data. 


SA seam effect: 


e There are large spikes in the number of January starts and December terminations. As in the 
case of EI, the SA could reflect a real phenomenon. After all, SA starts and terminations are 
affected by labour market conditions, in the same way EI starts and terminations are affected. 


e However, there are at least two reasons why it is reasonable to conclude that to a large extent 

the SA spikes represent a seam effect: 

a) the SA spikes are much larger than the EI spikes; and 

b) the monthly pattern appears suspicious: for example, the majority of respondents showing 
a December 1996 termination of SA benefits are reporting benefits in all the months in 
1996 and no benefits in any of the months in 1997; the suspicion is that when individuals 
receive SA in most months of the year, they tend to say that they received SA in all the 
months; conversely, when they do not receive SA in most of the months of the year, they 
tend to report that they did not receive SA at all. 


3. Recommendations 
a) Employment Insurance 


The quality of the EI data in SLID is fairly good: there is a high reporting rate, and no evidence 
of seam effect. Therefore, there are no overwhelming reasons for any drastic measures here. 
However, some of the changes recommended below with respect to SA would improve further 
the reporting of EI. 


b) Social Assistance 


By contrast, SA is subject to significant under-reporting of benefits and pronounced seam effects. 
A number of steps can be taken to improve the reporting of SA: 


¢ Imputations: One way of addressing the under-reporting problem is by imputing months and 
benefit amounts based on the personal characteristics and labour force status of the 
respondent. 


e Data linking: A more satisfactory approach is to expand the concept of data linking and link 


SLID not only to income tax data, but also to provincial SA records (the same suggestion 


applies to EI data, although the urgency is less than in the case of SA). Data linking would 
also deal with the seam problem. 
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e Dependent interviewing: At the present time, the dependent interviewing questions with 
respect to SA (and EI) are not truly dependent interview questions. They are simply used to 


put the respondent in the right frame of mind, but are not being used to improve the quality of 
the data: 
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a) 


b) 


in the case of those who reported receiving benefits in December of the previous year, if 
the information is contradicted in the next year no action is taken; and 

in the case of those who did not report receiving benefits in December of the previous 
year, no dependent interviewing questions are asked at all. 


It will make sense to: 


a) 


b) 


apply dependent interviewing questions to all respondents—i.e. even if respondents 

reported in January 1997 that they did not receive any SA make sure they agree with the 

information they provided last year about benefits in December, regardless if they said 
they had received benefits in December or not; 

if there is a conflict in the dependent interviewing questions, an attempt should be made 

to resolve the conflict—both during the interview (through further probing) and through 

editing of the responses later on; this is particularly important if the conflict in questions 
can lead to a seam effect; for example: 

- suppose that respondents reported in January 1997 that they received SA 
benefits in December 1996; 

- then, suppose that in the next interview (on January 1998) they dispute that 
they received SA in December 1996 and also report no SA in 1997; in the 
absence of no resolution of the two conflicting answers, it would appear that 
an SA claim was terminated in December 1996 (causing a seam effect). 


Income editing: With respect to the adjustment and imputation, given that there is an under- 
reporting of EI (as well as SA and WC), it will make sense to be more generous. In particular, 
if there is evidence from any source (January interview, May interview, or tax return) that 
respondents received EI, then it should be assumed that they received EI. Consequently, the 
amount of benefits and monthly pattern of benefits if missing should be imputed. 
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1. Introduction 


There is considerable policy interest on the impacts of the recent changes in Employment 
Insurance (EI) and social assistance (SA). The Survey of Labour and Income Dynamics (SLID) is 
particularly well suited for this type of analysis, because it follows a panel of individuals 
throughout the recent changes (1993-97). 


The SLID data have not yet been used extensively to study topics related to EI and SA. There are 

two potential concerns about the quality of the SLID data: 

e Under-reporting: Virtually all surveys that deal with government transfers (e.g. Survey of 
Consumer Finances) under-report government transfers. Particularly concerning is the 
possibility that the extent of under-reporting may change from year to year, or that it may 
relate systematically to respondent characteristics. 

e "Seam" effect: A "seam effect" arises from a tendency to incorrectly report receiving benefits 
throughout the year, while in fact benefits may have been received only in certain months, 
leading to artificial spikes in the January starts and December terminations of benefit spells 
(seam effect); this problem distorts the length of benefit spells and the sequencing of different 
types of benefits. It is known from other longitudinal surveys that respondents may report 
receiving a government benefit throughout the year while, in fact, they may have received it 
only in certain months. A similar problem has been found in SLID with respect to job spells 
(see Cotton and Giles, 1998). 


The above situation would lead to artificial spikes in the number of January starts and December 
terminations of benefit spells. At the same time, it would distort the length of benefit spells, as 
well as the sequencing of different types of benefits (EI and SA in this case). In order to correctly 
assess the interaction between EI] and SA, it is important that we are confident that the length and 
sequence of benefit spells is being correctly recorded in SLID. 


The objectives of this study are to assess the extent of underreporting and "seam" effect in the 
reporting of EI and SA in SLID and propose ways of overcoming them. 


We begin by examining the accuracy of the reporting of EI in SLID, first the issue of under- 
reporting (Section 2) and then the issue of seam effect (Section 3). Then we examine the same 
two issues with respect to SA (Sections 4 and 5). The conclusions are summarized in the 
executive summary. 
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2. EI Under-Reporting 


2.1 Analysis 


In this section we analyzed each annual cross-sectional sample, and compared the monthly 
incidence of EI benefits and the annual amount of EI payments to administrative data. Our 
comparators were the following:! 


e monthly number of EI beneficiaries (source HRDC’s EI data, available through CANSIM); 


e annual number of EI beneficiaries, i.e. number who received benefits in at least one month 
during the year (source Revenue Canada’s T1 file, available through their web site);” and 


e annual EI payments (source: HRDC and Revenue Canada; the Revenue Canada figures are 
lower than the HRDC figures because not all EI beneficiaries are tax filers). 


2.2 Findings 
(a) Monthly Number of EI Beneficiaries 


Chart 1A compares SLID estimates of the monthly number of EI beneficiaries to the number of 
EI beneficiaries reported by HRDC (in the 10 provinces only), while Chart 1B shows the ratio of 
SLID estimates to HRDC figures. The comparison shows that: 


e The monthly number of EI beneficiaries in SLID ranges from 75% to 100% of the number of 
beneficiaries reported in the HRDC administrative data. 


e Both the HRDC data and SLID data have a seasonal pattern. However, the SLID pattern is 
flatter than the HRDC pattern. As a result, the low points of the HRDC pattern come close to 
the SLID pattern and, at those points, the monthly number of EI beneficiaries in SLID comes 
very close to the HRDC figures. 


In summary, over the period 1993-97, the number of EI beneficiaries in SLID is under-reported 
by about 15%. The extent of under-reporting varies by month. However, on an annual basis, the 
extent of under-reporting is fairly stable over time. 


' Our analysis followed these steps: 
a) For each year, we selected respondents with positive cross-sectional weights for the corresponding year and 
complete labour market information (ILBWT26>0). . 
b) Then we weighted responses using each years cross-sectional, complete labour information weights 
(ILBWT26). ae 
c) Weused variables COMPTYPE and MTHRCV14 to identify in which months an individual received EI. 
d) Finally, we used variable UIBEN42 to determine the annual incidence of EI benefits and the amount of EI 


benefits received in a year. 


2 ‘The Revenue Canada data tend to understate the incidence of EI since not every El recipient files an income tax 
return. One could estimate the annual incidence of EI from the HRDC data; however, these data are not 


published regularly and will require analysis of the EI microdata (EI Status Vector file). 
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Chart 1A: Monthly Number of El Beneficiaries 
(SLID Estimates vs. HRDC Admin Data) 
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(b) Annual Number of EI Beneficiaries 


Charts 2A_and 2B compare SLID estimates of the annual number of EI beneficiaries to the 
number of income tax filers reporting EI income. According to this comparison, SLID fares very 
well. The reporting rate was 93% in 1993 and has risen to 105% by 1997. 


The high reporting rate is largely explained by the fact that SLID respondents are asked for their 
permission to link their questionnaire to their tax file. The percentage of respondents who 
consent has increased over time, and it stands now at about 75%. 


The fact that the reporting rate (vis-a-vis the tax data) has exceeded 100% may reflect the fact 
that some EI beneficiaries have not filed a tax return. Also, it may be related to the introduction 
of the second SLID panel in 1996. 


(c) Annual EI Payments 


Charts 3A and 3B show that the reporting of EI benefits relative to the income tax data is around 
100%, similar to the reporting rate observed in the previous section with respect to the annual 
number of EI beneficiaries. Relative to HRDC’s EI data, SLID’s reporting rate of EI benefits is 
about 90% 


Chart 2A: Annual Number of E! Beneficiaries 
Income Tax Data vs. SLID Estimates 
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Chart 2B: Annual Number of El Beneficiaries 
(SLID Estimates as a % of Income Tax Estimates) 
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Chart 3A: Annual El Payments 
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Chart 3B: Annual El Payments 
SLID as a % of HRDC Data & Income Tax Data 
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2.3 Conclusion 


The annual incidence of EI and annual EI payments are fairly accurately represented in SLID. So 
it may not be necessary to make any adjustments here. 


The monthly incidence presents a bit of a problem because the under-reporting is higher (about 

15%) and it also varies by month. The probable causes of the problem are: 

(a) respondents may report EI, but may not always be sure in which months they received EI; or 

(b) information about the receipt of EI is obtained from the tax file of the respondent, but there is 
no corresponding information of the monthly pattern in the January interview. 


Among the suggestions that could be considered: 

e With respect to annual incidence of benefits and beneficiaries there is no serious problems 
and no action is required. 

e With respect to the monthly incidence, the rate of under-reporting is rather low (about 15%) 
and has remained stable over time. We have two options: 

(a) The first option is not to take any action at all and use the SLID data as they are. 

(b) The second option is to impute a monthly pattern for those who received EI during the 
year, but for whom the monthly pattern of benefits is not known. If we take the extreme 
approach of assuming that these individuals received benefits in all months, then the 
monthly incidence will exceed substantially the actual monthly incidence (Chart 4). 
Therefore what is required is a more careful imputation to close the gap between the 
current SLID monthly pattern and the actual pattern.” 


3 The imputation methodology can be fine-tuned by linking SLID records to EI records (using name, birthday and 
address information); the linking would be used strictly for methodological reasons (to test the imputation logic) 


and no permanent analytical file will be created. 
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Chart 4: El Beneficiaries: SLID Estimates vs. Admin Data 
(After Treating "Do not Know" Cases as Receiving El) 
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3. EI "Seam" Effect 
3.1 Analysis 


In this section we analyze the sample of longitudinal respondents, in order to examine the 
monthly pattern of starts and terminations of EI spells. We want to see whether in fact there is a 
tendency for a disproportionate number of EI spells to start in January or end in December, and 
how extensive is this problem.* 


3.2 Findings 


Chart 5 shows that there are significant spikes in the number of January starts of EI spells. At the 
surface, this finding suggests the presence of a seam effect. However, examination of the 
establishment of new EI claims based on HRDC admin data also shows similar spikes. 


We followed the following steps: 

(a) We selected respondents with positive longitudinal weights (ILGWT26>0) and within scope (RESP99=1) 
in all five years (1993-97), who received EI or SA in any of the five years. 

(b) We weighted responses using the 1997 longitudinal weights. 

(c) We used variables COMPTYPE and MTHRCV 14 to identify in which months an individual received EI. 

(d) We defined the start of an EI spell as a month with EI benefits preceded by a month with no EI benefits. 


(e) Similarly, we defined the end of an EI spell as a month with no EI benefits preceded by a month with EI 
benefits. 


an 
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Based on this comparison, we conclude that the seam effect is not a serious issue with EL In 
other words, the spike of EI starts in January reflect a real phenomenon: a jump in the number of 
EI claims following the increase in temporary employment related to the Christmas holiday.” 


Chart 6 shows the monthly pattern of EI spell terminations. It shows that there is a spike in 
December (relative to the number of starts in the previous November of following January). This 
could be explained by the rise in seasonal jobs in December. However, because administrative 


numbers to compare with are not readily available, it is not clear to what extent the December 
spikes represent a seam effect or a real phenomenon.° 


In any case, there are even bigger spikes in EI termination in other months. So in this sense, the 
December spikes are of relatively less concern. Whether all these spikes are real or not is difficult 
to judge unless one produces similar statistics from the EI administrative data. 


Chart 5: El Benefit Starts: SLID vs. HRDC Estimates 
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> However, although the number of January starts in SLID is quite close to the actual number of starts, this is not 
the case for other months. In fact, for some months, SLID estimates are as low as half the number of starts of 


new claims according to HRDC records. 


6 The required data can be produced from the HRDC administrative microdata. 
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Chart 6: El Benefit Terminations: SLID Estimates 
(No comparison to administrative data available) 
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3.3 Conclusion 


There does not seem to be a seam effect with respect to EI starts, and it is unclear whether there 
is one with respect to EI terminations. 


Of more concern is the fact that the SLID pattern of EI starts deviates for certain months from 
the administrative counts. It is quite possible that the same problem exists with respect to EI 
terminations as well (but we do not have the necessary administrative data to test this). 


Among the suggestions that could be considered: 


@ With respect to the seam effect, there is no evidence that it is present and, therefore, no action 
is required. 


@ Possibly, some imputations may be desirable to fine tune the overall monthly pattern (since it 
appears to deviate from the actual pattern in certain months).’ 


7 For example, in the case where an EI spell appears to start in January, if the individual is still working in the 
same job in January, the start of the EI spell could be pushed after the end of the job spell. 
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4. SA Under-Reporting 
4.1 Analysis 


In this section we conducted the same type of analysis as we did for EI in Section 2. We compare 
the SLID to administrative data as follows: ® 


e The number of individuals reporting SA was compared to the monthly provincial SA 
caseload. Typically one person within the family reported SA to SLID. So, in this sense, the 
two data sets are comparable. 


e SLID estimates of total annual SA benefits were compared to provincial estimates of annual 
SA payments. 


The analysis of the under-reporting issue (whether it refers to cases or benefit payments) is 
problematic because the concept of SA is not as clear as the concept of EI. 


e In the case of EI, there is a single national program and there is no ambiguity whether a 
person receives EI or not. 


e By contrast, SA programs differ among the provinces. Moreover, there is ambiguity with 
respect to the nature of some programs. 


4.2 Findings 


In this section we compare the SLID estimate of SA applicants (i.e. individuals who reported 
SA—one individual per family unit) to the provincial caseload (one applicant per family unit). 


The analysis shows that there is extensive under-reporting of the number of SA applicants in 
SLID (Charts 7A_and 7B): The ratio of SLID estimates to the actual number of cases is just over 
50%. 


One factor behind the above result is that the SLID estimate excludes respondents who received 
SA during the year but did not report in which months they received SA. However, even after 
assuming that those who received SA in a year received it in all 12 months, the reporting rate is 
still significantly below 100% (it rises from 50% to 70%). In other words, even if we overstate 
the reporting of SA (by assuming that those who received SA in a year received it in all months 
of that year), we are still significantly below the number of cases reported in provincial 
administrative data. 


8 We used the following variables to identify receipt of SA: 
a) COMPTYPE and MTHRCV14 to identify in which months an individual received SA; 
b) SAPIS42 to determine the annual incidence of SA benefits and the amount of SA benefits received 
in a year; and 4 
c) Because SA is a family-based program (unlike EI which is strictly based on the individual’s insurability) 
we also used variable FMSAV27 (from the DEC31FAM entity); this variable indicates in which months 


any member of the family received SA. 


® For example, Alberta’s "Program for Independence" and "Children in Need" are income tested but it is not clear 
whether they should be considered as part of SA. Equally important, it is not clear how respondents think about 
such programs—i.e. whether they think of them as SA or not—which of course will affect their responses. 
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Similarly, with respect to annual payments there is considerable under-reporting of SA in SLID. 
Chart 8 shows that the ratio of annual SA payments according to SLID, relative to the 
administrative data, is about two-thirds. 


Chart 7A: Number of SA Applicants (one per family unit) 
SLID Estimates vs. Provincial Caseload (one per family unit) 


1,800,000 
1,600,000 ____ Provincial Admin: Cases 


1,400,000 
SLID (if months not known assum 
1,200,000 DO0COOP0000 


1,000,000 
800,000 
600,000 


SLID (exclude if months are not known) 


400,000 


200,000 


0 


Jan-93 
Apr-93 
Jul-93 
Oct-93 
Jan-94 
Apr-94 
Jul-94 
Oct-94 
Jan-95 
Apr-95 
Jul-95 
Oct-95 
Jan-96 
Apr-96 
Jul-96 
Oct-96 
Jan-97 
Apr-97 
Jul-97 
Oct-97 


Chart 7B: Number of SA Applicants (one per family unit) 
SLID Estimates/ Provincial Caseload 
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Chart 8: SA Benefit Payments 
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4.3 Conclusion 


There is a significant under-reporting of SA in SLID (both in terms of monthly incidence and 
total amount of benefits received). The precise size of under-reporting depends on the 
comparator. The comparators used here were provincial data of caseload and benefit payments. 


The above finding is not perhaps surprising since the same problem is common to the Survey of 
Consumer Finances. Part of the reason for the under-reporting of SA is the confusing nature of 
social assistance. Another reason may be that there is more stigma attached to receiving SA and 
greater reluctance to report it in a survey. 


It should also be added that linking SLID to tax data is somewhat less effective in the case of SA 
than in the case of EI, since it is more common for SA recipients than EI recipients not to file an 


income tax return. 


There is no easy solution to the above problem, short of linking SLID records to provincial SA 
records (in the same way SLID records are linked to tax records). Imputing a monthly pattern, for 
those who we know received SA but the months are not known, will improve the monthly 
incidence of SA. However, it will still remain significantly below administrative counts, since on 
an annual basis there is also considerable under-reporting. 
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5. SA Seam Effect 
5.1 Analysis 


In this section, we performed similar analysis to that performed for EI in Section 3, to see if 
January starts and December terminations of SA benefit spells are relatively more common. 


5.2 Findings 


The results of the analysis show that there are very significant spikes in December terminations 
and January starts of SA spells. The spikes were equally dramatic regardless if we looked at the 
number of SA applicants or the number of SA beneficiaries (i.e. individuals living in a family 
that one member is a SA applicant): 


e Chart 9A shows that there are pronounced spikes in the number of January starts of SA 
applicant benefit spells. 


e A small consolation is that the spikes have declined over time. In 1993 January starts were 
about eight times as common as other months in the year. By 1997, the ratio declined to 
about four. 


Charts 10A_and_1OB show that there are similar spikes with respect to SA terminations. 
However, contrary to the case of starts, there is no reduction in the spikes over time. 


Chart 9A: SA Applicant Starts 
(Individual him/herself started receiving SA) 
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Chart 9B: SA Beneficiary Starts 
(Someone within the family start receiving SA) 
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Chart 10A: SA Applicant Terminations 
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Chart 10B: SA Benefit Terminations 
(Family Received Benefits) 


250,000 

200,000 

150,000 

100,000 

50,000 
0 
C06 62.8 OO De PS SS 10 OO OT Mio wc Io) ame eEEOS 
& S.3. 8) 82S 308. 872306 Us alo fo oem 


5.3 Nature of the SA Spikes 


Are the above spikes proof of a seam effect with respect to starts and terminations of SA spells? 
It is impossible to be absolutely sure since we do not have administrative data of starts and 
terminations of SA spells. 


In the case of El, where administrative estimates of starts of new benefit spells were available, 
we found that the SLID spikes represent for the most part a real phenomenon, rather than a seam 
problem. 


It is possible that this is also the case to some extent with SA spells. After all, SA starts and 
terminations are affected by labour market conditions, in the same way EI starts and terminations 
are affected. 


However, a closer look at the raw data suggests that there is a problem with the way SA data are 

reported in SLID. For example (see Chart 11): 

(a) In December 1996, there were 199 SA terminations—i.e. 199 individuals lived in families 
that received SA in December 1996, but not in January 1997 (117+15+56+11). 

(b) In 117 of the above cases (59%), the individuals reported benefits in all the months in 1996, 
and no benefits in any month in 1997; these cases appear highly suspicious; one possibility is 
that many of these individuals received SA in most months in 1996, but not necessarily in 
December 1996. 

(c) In 56 of the above cases (28%), the individuals reported benefits in some months in 1996, but 
no benefits in any month in 1997; these cases appear somewhat suspicious, although less 
suspicious than the 117 cases above; some of these individuals may have received benefits in 
January 1997; if there was no link to their tax file (or they did not report SA in their tax 
return) they would still appear as not having received SA in 1997. 

(d) The remaining 26 cases (15+11) appear legitimate. 
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Based on the above analysis, 117 to 173 of the 199 terminations in SA in December 1996 (i.e. 
59% to 87%) are highly or at least somewhat suspicious. We showed in Chart 10B that the rate of 
SA terminations in December 1996 was about 4 times as high as for the rest of the months. To 
attribute this spike entirely to incorrect reporting would require that 75% of the records be 
suspicious (which is within the range of 59% to 87% mentioned above). 


Chart 11: Comparison of Monthly Receipt of SA: 1996 vs. 1997 


(Unweighted sample of individuals living in families that reported SA in December 1996) 


No. % 
1997: received SA in all months 719 60.6% 
1997: SA in January, but not in all months 61 5.1% 
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5.4 Proxy Responses 


More than half of the responses in SLID are obtained through proxy—i.e. the same member of 
the household provides information for all the members of the household. The individual that 
reports on behalf of the household is not necessarily the same person each year. If the individual 
reporting changes between two surveys and the two members of the household have a different 
understanding of what is SA, this would lead to a seam effect. For example, a household that was 
on social assistance for two years, may appear as exiting from SA at the end of the first year if 
the person reporting in the second year does not think the family is receiving SA. This problem is 
more likely to happen with SA than other programs since, as it was mentioned earlier, the 
definition of SA is not very clear. 


5.5 Logit Regression Analysis 


We also conducted logit regression analysis of the January starts and December terminations to 
see if the observed spikes are systematically correlated with personal characteristics.’° The 
regression results are reported in the Appendix. They show that virtually none of the coefficients 
are significant at the 95% level. In other words, although the spikes are significant, they are not 
systematically correlated with any personal characteristics and, in that respect, are more random 
in nature. 


5.6 Dependent Interviewing 
a) Introduction 


SLID uses dependent interviewing, a technique that consists in reminding respondents of 
information that they supplied in the previous interview, to assist them in recalling events that 
have occurred since. 


A survey such as SLID can give rise to “seam problems,” since respondents sometimes have 
trouble remembering the dates on which they started work, stopped work, experienced an 
interruption of work, etc. over a one-year period. These memory errors can result in an excessive 
proportion of periods beginning or ending on the “seam” of two consecutive reference periods. 


Reminding the respondent of information collected in the previous interview can help reduce 
these seam problems. This technique is much easier to use in a Computer Assisted Interview 
(CAT) environment. CAI also serves to detect errors of logical consistency between the 
information collected in the labour interview and, a few months later, the information collected 
in the income interview. 


10 Ware ; 
The universe for the analysis of SA starts, for example, was individuals with a SA start in any month of the year; 


in this case, the dependent variable was one if individuals had a J anuary start, and zero otherwise. 
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b) Dependent Interviewing for EI, SA and Workers Compensation 


We concentrate here on EI, although the same points also apply to SA and Workers 
Compensation: 


If in last year’s survey (Janu 1997 interview) respondents reported that they received EI in 


December 1996, then in the January 1998 interview they are asked the following dependent 
interviewing questions: 


e Based on our interview of a year ago, respondent received EI in December 1996. Is this 
correct? 


If it is not correct, no retroactive change is made to the SLID information; basically the new 


information is ignored. 


Regardless of what answer was given above, respondents are then asked: 
e Did respondent receive any income from EI in 1997? 
@ (if answer is yes) in which months? 


If in last years survey (January 1997 interview) respondents did NOT report that they received EI 
in December 1996, then no dependent interviewing takes place. 


c) Adjustments and Imputations 


Again, the discussion about adjustments and imputations applies equally to EI, SA, and Workers 
Compensation. 


SLID is linked to the tax return: 
In the case that respondents have consented to the linking of their SLID record to their tax return, 


then: 
the amount of EI benefits in 1997 is obtained from their 1997 income tax return; 
if in the January 1998 interview they reported that they did not receive EI, then the 12 months 
are recoded from 2 (no) to 7 (received EI, but months not known). 
conversely, if in January 1998 they said that they received EI but no EI was reported in their 
tax return, the months are recoded from 1 (yes) to 2 (no EI benefits). 


SLID is NOT linked to the tax return: 
the amount of EI is obtained during the May 1998 interview; 
if they did not report El benefits in the May 1998 interview, but in the January 1998 
interview they reported that they received EI in some months in 1997, then the EJ amount for 


1997 is imputed. 
conversely, if an amount is reported, but they said in the January 1998 interview that they did 
not receive EI in 1997, the month codes are changed from 2 (no) to 7 (received EI, but 


months not known). 
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d) Comments 


With respect to the dependent interviewing questions, the above listed questions are not truly 

dependent interview questions. They are simply used to put the respondent in the right frame of 

mind, but are not being used to improve the quality of the data: 

@ in the case of those who reported receiving benefits in December of the previous year, if the 
information is contradicted in the next year no action is taken; and 


e in the case of those who did not report receiving benefits in December of the previous year, 
no dependent interviewing questions are asked at all. 


It will make sense to: 


e apply dependent interviewing questions to all respondents—i.e. even if respondents reported 
in January 1997 that they did not receive any EI make sure they agree with the information 
they provided last year about benefits in December, regardless if they said they had received 
benefits in December or not; 


e if there is a conflict in the dependent interviewing questions, an attempt should be made to 
resolve the conflict—both during the interview (through further probing) or through editing 
of the responses; this is particularly important if the conflict in questions can lead to a seam 
effect; for example, suppose respondents said last year that they received EI benefits in 
December 1996, but then next year they dispute this and also report no EI in January of 1997; 
in the absence of resolution of the conflicting answers, it would appear that an EI claim 
terminated in December 1996 (causing a seam effect). 


With respect to the adjustment and imputation, given that there is an under-reporting of EI (as 
well as SA and WC), it will make sense to be more generous. In particular, if there is evidence 
from any source (January interview, May interview, or tax return) that respondents received EI, 
then it should be assumed that they received EI. Consequently, the amount of benefits and 
monthly pattern of benefits if missing should be imputed. 


5.7 Conclusion 


There are significant spikes in December terminations and January starts of SA spells in SLID. 
These spikes appear to reflect the presence of a significant seam effect. Among the possible 
solutions that should be considered are: 


e In the shorter term, certain adjustments could be made to the data. For example, in the case 
where an EI spell appears to start in January, if the individual is still working in the same job 
in January, the start of the EI spell could be pushed after the end of the job spell. 

e In the longer term, the dependent interviewing question could be strengthened, as suggested 
above. This suggestion would help reduce to size of the spikes in December termination and 
January starts. 

e Given the magnitude of under-reporting of SA and the magnitude of seam effects, it will be 


desirable in the future to also consider the idea of extending the concept of data linking 
beyond income tax data, and include EI records and provincial SA records. 
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Appendix: Logit Regression Results 


We used two different samples: (a) benefit start in 1996; and (b) benefit termination in 1996. 


The dependent variable was the logit of January starts (1 if started in January; zero otherwise) 
or the logit of December terminations (1 if ended in December; zero otherwise). 


The independent variables are shown in Box A. 


The key statistic that is relevant to our analysis is the Sigma statistic. If sigma is less than 


0.025, then the corresponding coefficient is significantly different than zero at the 95% level 
of confidence. 


e The results show that virtually none of the coefficients are significant. 
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Box A: Explanation of Logit Independent Variables 


AGE96 (0) Age 16-24 (omitted) 
AGE96(1) Age 25-44 

AGE96(2) Age 45-64 

AGE96(3) Age 65+ 

DISABS96(1) Had a disability in 1996 
HHSZ96(0) Household size 1 (omitted) 
HHSZ96(1) Household size 2 

HHSZ96(2) Household size 3 

HHSZ96(3) Household size 4+ 
HLEVEG96(0) Education not known (omitted) 
HLEVEG96(1) Less than grade 9 
HLEVEG96(2) Grades 9-13 

HLEVEG96(3) High school graduation 
HLEVEG96(4) Some university 
HLEVEG96(5) Non-university diploma 
HLEVEG96(6) University degree 
MARST96(0) Married/common law (omitted) 
MARST96(1) Separated, divorced, widowed 
MARST96(2) Never married 

PVRES96(0) Atlantic (omitted) 
PVRES96(1) Quebec 

PVRES96(2) Ontario 

PVRES96(3) Prairie 

PVRES96(4) Be 

SEX96(1) Female 

TTINC96(0) Under 15,000 

TTINC96(1) 15,000-24,999 

TTINC96(2) 25,000-39,999 

TTINC96(3) 40,000-59,999 

TTINC96(4) 60,000-79,999 

TTINC96(5) 80,000 plus 
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1. SA JANUARY STARTS: INDIVIDUAL RECEIVED BENEFITS 
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2. SA JANUARY STARTS: ANY FAMILY MEMBER RECEIVED BENEFITS 


Total number of cases: 


-2 Log Likelihood before: 
-2 Log Likelihood after: 


Variable 


AGE96 
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